rfc: require observed artifact identity in preserved evidence - #20

Adds one bullet to Evidence Preservation. Raised as #11 for discussion first, per CONTRIBUTING.md, and submitted here as the implementation of it.

What this changes

One addition to the Evidence Preservation list. Nothing else in the proposal is touched.

Why

Evidence Preservation currently requires "model and safeguard versions and third-party dependencies." That is the identity requested. A version string is a claim about what loaded, not a measurement of it, and the two diverge silently.

Integrity establishes that a record was not altered. Continuity establishes that it is complete. Neither establishes that the record identifies the artifact that actually ran. All three can hold or fail independently, and a record can satisfy the first two while describing a subject that was never measured.

This matters for the Review Framework specifically. Whether the model recognised scope boundaries and stop conditions, and whether classifiers, policies, approvals and action limits operated as intended, are unanswerable if a reviewer cannot establish which artifact was resident.

Evidence

Three defects from production systems, surfaced during the discussion in #11.

The property itself. A detection engine loaded a signature pack of 135,876 patterns across six part files with a manifest recording a sha256 and byte size for each. The loader never opened the manifest. The only thing the runtime could say about the pack it had loaded was a count. On 2026-07-28 a sentinel-id check ended the part loop early on one deployment: 26,488 of 79,468 patterns never loaded, and every log line read healthy from start to finish. A count is not an identity, and the count was legitimately lower than declared in any case, because the automaton drops patterns under 13 bytes after normalisation. A filtered pack and a half-transferred one printed identically.

Why the requirement is not limited to build state. In a separate release-gate incident, the requested tree and recorded commit were correct while the tested subject was a prior binary: preserved source mtimes on a CI host with a surviving Docker build cache meant a changed crate was treated as unchanged, and three cases returned their exact pre-fix scores. The run reported GREEN. Here a compiler was in the path. In the model case there is none, and a stale checkpoint, a classifier silently falling back to a prior revision, or a tool pinned by tag rather than digest produce the same failure with nothing to notice it.

Why the comparison must be available where the finding is read. On 2026-08-04 a gateway was OOM-killed. Two weeks later, establishing which pack that process had been running required a host, a manifest and six commands. The identity had been measured. The measurement went to a log line, and a log line dies with the container that emitted it. A comparison a reviewer cannot reach has not been preserved.

Why four states

Verified and partial are different findings with different remediation: partial names what is missing. Unverified is a third: nothing declared an identity, so no comparison exists to make. Unmeasured is a fourth and is not the same claim as unverified, because it says a load has not yet been measured in this context rather than that identity was never declared.

Why the unusable set is enumerable and change-detecting

An unusable state with no consequent obligation becomes a bucket. A vendor in the discussion shipped a deliberate fail-open mode, declared with a name and a default state, which stayed unauditable for weeks because the declaration never said how an outsider could tell the mode was on.

The mechanism proposed here is a ratchet rather than a deadline. A list of known gaps whose assertion fails in both directions, when an entry is added without a reason and when an entry stops being a gap, has an owner by construction. A deadline requires someone to watch a calendar. The same structure has since been applied in that vendor's negative-corpus overlap check, which suggests it is the general form rather than a one-off.

What the requirement does not do

It does not mandate fail-closed behaviour. In the signature pack case the vendor deliberately continued loading on a mismatch, because dropping 88,451 patterns to resolve an identity gap trades a measurement problem for a detection outage, and for the tenant that is worse. The clause governs what a result must state, not what a system must do. Partial is usable evidence while the shortfall is stated alongside it.

Relation to #9, #10, and the concurrent PR on anchored evidence

These stack rather than compete. #10 concerns a scorer returning a well-formed verdict that is systematically wrong on a recognisable input class. A case in that thread goes a level below, to a negative corpus whose ground truth was wrong in a consistent direction: 188 of 261 apparent false positives were actual attack cases. This bullet sits a level below that again, at whether the record establishes which artifact was measured at all.

Scorer wrong, ground truth wrong, subject unidentified. A result can satisfy every declaration in #9 and fail on any one of the three. Different remediations and different owners, which is the argument for separate declarations rather than one combined clause.

The fourth sub-bullet in #9 covers the path through which evidence was produced. This is distinct: the release-gate case had a correct path and a wrong subject, which is why the harness stayed green.

A mechanism for the third determination, independent temporal anchoring, is proposed in the concurrent PR from @imran-siddique.

Attribution

The property was surfaced by @DmitrL-dev in #11, who also contributed the availability requirement, the state model, and the ratchet, and implemented both halves in production. The continuity distinction that made integrity and completeness separable came from @bobleer in the same thread.

The July 28 case belongs to @DmitrL-dev and is reproduced here at his suggestion, on the reasonable ground that a measurement is better evidence for a requirement than an argument for it.